System And Methods For Transit Path Security Assured Network Slices

ABSTRACT

Systems and methods of configuring, managing and ensuring security compliance of Virtual Network Slices that transit through physical networks, virtual networks (SDN), cloud networks, radio access networks, service provider networks, and enterprise networks are identified. The methods include user side security validation methods while attempting to use a network slice for a specific service, and security validation of physical or virtual networks and the associated transit network elements. The methods disclose enriching the Security Certificates with policy parameters and the associated procedures that transit elements are required to assure for security compliance. Additionally, methods for incorporating a mobile native security platform in Wireless Mobile Network (4G/5G) that supports generating X.509 Certificates enhanced with policy requirements, validating allowed/disallowed list of transit network vendor devices, virtual network appliances are identified.

This application is a continuation of U.S. patent application Ser. No.17/321,405 filed May 15, 2021, which claims priority of U.S. ProvisionalPatent Application Ser. No. 63/025,744 filed May 15, 2020; and63/049,151 filed Jul. 8, 2020, the disclosures of which are incorporatedherein by reference in their entireties.

BACKGROUND

Public Key Infrastructure (PKI) uses X.509 Digital Certificates thatfacilitate validating identity and trust of entities, such as hardwaredevices, Operating Systems, Applications and others, for establishingsecure communications. PKI uses Public and Private Keypairs associatedwith an entity (such as a first entity), where only the public key iscommunicated with a remote entity (referred to as the second entity,which may be, for example a web server), and when communication isencrypted with the public key, only the first entity that is holding theprivate key can decipher the encrypted information. PKI uses a pluralityof infrastructure components such as CA (Certification Authority), RA(Registration Authority), TA (Trust Anchor) and others, for validatingthe authenticity of the Digital Certificates. Using PKI methods is wellknown and extensively used for secure communication over the internet.Examples include IPsec (Internet Protocol Security), TLS (TransportLayer Security), and others. As shown in FIG. 2, X.509 Certificatesinclude several identifiers, that when combined, uniquely identify thecertificate, key algorithms, subject (identity of the certificateowner), serial number, certificate issuer, validity periods (start time,end time), certificate management and revocation information, andcertificate policies.

End-systems, protocol terminating endpoints and applications such asclient applications, servers and other resources that use an encryptionprotocol (such as IPsec, TLS and others) to secure communications. Whenestablishing a trusted communication path, each endpoint validates thetrust of the remote endpoint using PKI methods that use DigitalCertificates. Also, endpoints that terminate protocols or offer servicesvalidate identity and trust of the other using Digital Certificates andPKI methods even if such communication is not encrypted. The twoendpoints typically communicate through a plurality of local, private orpublic networks (for example Internet, public cloud, private cloud, SDN)of multiple operators or service providers. These communications passthrough many physical or virtual network switches, Layer 2 bridges,Layer 3 routers. Transit networks forward user data packets based onLayer 2 or Layer 3 addresses within the packets using source learning(MAC Bridging), or routing (L3 Routing) or forwarding paths orchestratedby a network controller (e.g., SDN controller). If the transit networkuses Virtual Network Functions (VNF), there may be multiple layers ofnetworking, for example, using VXLANs or GRE tunnels. While the abovedescribes secure communication, other resources may also make use ofthis concept.

Enterprises are migrating to SDN, SD-WAN, and Cloud Environments usingVirtual Networks, Virtual Servers, and Commodity platforms that areshared by multiple tenant enterprises, for many reasons. These includegreater flexibility, growth, reduced operational cost, transportingclassified and highly sensitive data. In these scenarios, it isessential that every physical and virtual network element guaranteespolicies that do not compromise the security of the data beingtransported.

Further, it is important that the physical and network elements are froman approved (trusted) source consistent with the policy of use. This isto assure that these network elements (NEs) will remain available duringtimes of crisis and not subject to disruption if the provider (OEM) ofthe network element is an adversary potentially using a maintenancebackdoor to intentionally create the disruption as a tool of war.

As enterprises that handle sensitive information move to virtual/cloudnetworks, and applications move to mobile networks for anytime access,the networks that carry the sensitive data are shared on the samephysical resources with layers of virtualization. Security compromisesin the shared network, or bad actors may get access to or get copies ofclear or encrypted transit traffic. While endpoints, such as client andserver, may encrypt data during transfer through the network, with everincreasing compute resources (such as CPU, Memory, Storage, Network),the data may be copied in the transit shared network and decrypted. Anendpoint that participates in control plane or application protocolswhose trust/identity is not assured could copy or direct flows tomalicious bad actors. Malware and bad actors may take advantage of thesecurity holes and deficiencies in the network, the computing devicesand the software. In many cases, these security holes are dependent onspecific vendor hardware/software or specific version and model numbersor where they are manufactured, distributed, or installed. The prior artrouting and forwarding methods use destination L2 or L3 addresses ateach physical or virtual network hops without consideration of devicevendors, models, manufacturing locations etc. Additionally, enterprisesthat use the transit networks for transporting highly sensitive data donot have visibility or control over which devices or paths (locations),their sensitive data traverses through. While Layer 3 routing methodsinclude source routing where a source dictates the route that itspackets traverse, these routes specify IP Addresses and not virtual orphysical device provenance data. Restricting network paths, and physicalor virtual network functions (VNFs) that participate in control andapplication protocols and provide services to physical and virtualnetwork, computing and storage elements whose trust could be assuredfrom start to operational use via Digital Certificates significantlyreduces security compromises.

5G Architecture teaches the concept of “Network Slices” (NS), by whichan operator offers network services with different Quality of Service(QOS) such as bandwidth, delay, priority etc., on the same physical andvirtual network, compute, memory and storage resources for differingvertical application uses, such as IOT, Healthcare, V2X and others. Thenetwork slices may span across an operator's access and core networks.The network slices facilitate offering Macro and Micro services withdifferent QOS requirements on the same infrastructure. The creation andmanagement of network slices is specific to the operator.

Therefore, it would be beneficial if there were a system and methodsthat extended the concept of “Network Slices” to improve the trust andsecurity of communications.

SUMMARY

The present application describes “Secure Network Slices”, which arelogical networks between a plurality of physical or virtual endpoints(VNFs) such as client applications, application servers, databaseservers, proxies, protocol endpoints and enterprise edges where thetransit physical or virtual network elements are trust assured byenhanced X.509 Digital Certificates. The certificate enhancementsinclude but are not limited to provenance metadata attributes, such asthe Name of the manufacturer, Country and Geographical Region, permittedand prohibited list of manufacturers, manufacturing locations, modelnumber, version/revision number etc. User applications, such as IPsecand TLS, may establish secure encrypted sessions between clients andservers over the SNS logical networks. While setting up and managing the“Secure Network Slice” (SNS), every transit network element that ispermitted to carry traffic is validated to assure that it holds theenhanced digital certificate per the required security policies. Thetransit 5G Access and Core network that carries the SNS may includemultiple network domains by multiple operators, Public and PrivateClouds, Software Defined Networks (SDNs), private networks and others.Additionally, when the underlying network is reconfigured due to networkfailures, load balancing of VNFs, or other reasons, the SNS providerassures that the re-routed traffic transits through trust validated NEsor revalidates the trust of NEs. The methods identified herein may beincorporated as layered software modules within physical or virtualnetwork elements to create, manage, assure, monitor and validate securenetwork paths for macro and micro application services.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the present disclosure, reference is madeto the accompanying drawings, in which like elements are referenced withlike numerals, and in which:

FIG. 1A is an example network infrastructure where the methods disclosedherein may be deployed.

FIG. 1B is an example network infrastructure where the methods disclosedherein may be deployed, according to another embodiment.

FIG. 2 shows an example of a prior art X.509 Digital certificate withstandard attributes, such as Version, Certificate Serial Number,Signature Algorithm, Issuer (identity of the issuer), validation period,subject which identifies what the certificate is for, Public Key,Subject Alternative Name and other parameters.

FIG. 3 shows example metadata extensions to the X.509 Certificate, whichis termed an enhanced Digital Certificate (eDC).

FIG. 4 shows the configuration of a secure network slice according toone embodiment.

FIG. 5 shows an example deployment of a SNS Operations and ManagementPlatform (SNS-OMP) in a multidomain network interacting with networkcontrollers of each domain.

FIG. 6 shows the PKI infrastructure deployed according to oneembodiment.

FIG. 7 shows network dataflows of network slices through multiplenetwork domains.

FIG. 8 shows a flowchart showing the creation of a secure network slice.

FIG. 9 shows physical network path maps through multiple operators andcloud service providers.

FIG. 10 shows an example deployment configuration with the end to endMulti-domain security system components in an Amazon AWS CloudEnvironment, which may be an example of a multi-domain securitycomponent shown in FIG. 6.

DETAILED DESCRIPTION

The present disclosure extends the network slices defined in 5G toinclude additional trust and security policies in addition to QOSlevels. Additionally, the “Secure Network Slices”, disclosed herein areend to end, for example, between one geographical location of anenterprise to another geographical location traversing multiple operatornetworks.

The present disclosure defines additional metadata attributes, such asprovenance data which includes the country, region, specific locationwhere the subject of the certificate is manufactured, identity of themanufacturer, model, version, and other parameters, as shown in FIG. 3.The specific metadata attributes that an entity is required to supportfor enhanced trust are dependent on the specific enterprise and thedepartmental or operational use case.

As described above, the present disclosure is directed toward “SecureNetwork Slices”, which may be incorporated as layered software withinphysical or virtual network elements to create, manage, assure, monitorand validate secure network paths for macro and micro applicationservices. Embodiments of this may include:

-   -   (1) A layer of security that requires compliance of the transit        physical or virtual network elements that process sensitive data        of high security enterprises (such as the DOD, Financial        Services and others) based on metadata attributes in Enhanced        Digital Certificates (eDC). The security layer assures security        of the path by taking into account:        -   requirements of physical paths,        -   identity of the transit physical/virtual network elements,        -   their types,        -   vendor identification, and        -   additional policies managed by the Security Management            platform, type of network forwarding functions (L2 Bridging,            switching, routing), types of routing (source routing,            hop-by-hop routing), redundancy, service provider boundaries            and the associated enhanced digital certificates.    -   (2) Security Assured Virtual Routing—In a layered network        element (NE) that includes hardware, multiple layers of        software, cloud-network, and validation of allowed/prohibited        vendors, creating compliant Virtual/Physical interface port        groups, where each interface is allowed as a member based on the        next-hop network element that it connects to. If the transit        path traverses through multiple communication service providers        (CSPs) or Cloud providers, at the edge that connects two        providers, cross validation of certificates, and validating the        compliance of the specified requirements from both the service        providers is essential.    -   (3) Multiple policy profiles with different QOS primitives,        read-only, read-write, upgrade limits for each QOS primitives in        enhanced certificates, and plurality of authentication methods        that include 1 or more of:        -   a. PKI Certificates,        -   b. multi-step verification via independent paths,        -   c. voice/speaker identification,        -   d. SSO (Single Sign On), MFA (Multi-Factor Authentication),        -   e. OAUTH or others    -   (4) Enhanced Digital Certificate (eDC)—Metadata and Policy        enhancements to the digital certificates, that include vendor id        check for the transit NEs, virtual network elements, gateways,        bridges, servers and other devices.    -   (5) Enhancements in transit Network Element (NE) for supporting        secure network slices. Trust assurance of metadata and other        policies are validated when eDCs are incorporated into the        network element. Examples include providing information on eDC        compliance of a plurality of virtual/physical elements (for        example, virtual network layers, linecards and others) from SNS        logical network either in-band or via control and management        paths. Additional enhancements include the ability to determine        the mirroring or replicating interface traffic that carries SNS        traffic to an additional interface in addition to the        destination interface for the traffic.    -   (6) Client prescriptive physical and network paths and network        types that can ensure on request and in operation that the        chosen routing transits only over communications systems        technologies that are known and approved by the client. An        example is a US Govt agency requiring “SNS can only use a        dedicated connection path through allowed physical devices from        Vendor A, manufactured in USA”

Having defined various embodiments, a detailed description follows. FIG.1A is an example network infrastructure where the methods disclosedherein may be deployed. A multi-operator multi-domain network 100 mayinclude client devices 101, end systems, Access Networks 102, such aswireless base stations, WiFi Access Points and others, and a corenetwork 103. The Core Network 103 includes Accessibility and MobilityManagement Function (AMF), Session Management Function (SMF), User PlaneFunction (UPF), Unified Data Management (UDF), Policy Control Function(PCF), Authentication Server Function (AUSF), Network Slice SelectionFunction (NSSF), SDN, Public/Private Clouds, Operator Core Networks(5G), and Physical or Virtual Servers (VNFs). Further, various services104 may be connected to the network infrastructure, such as broadbandservice, voice call and text, low latency medical service, industrialIoT service, communication service and others.

FIG. 1B shows a second example network infrastructure where the methodsdisclosed herein may be deployed. In this embodiment, there areendpoints, which may be clients 105 or servers 106. Between theseendpoints, there are a plurality of transit network elements. Thesetransit network elements may include cellular infrastructure 107, suchas 4G or 5G devices, multiple carriers 108 and cloud providers 109. Incertain embodiments, a central broker 110 may be able to access each ofthese network elements. In this disclosure, the central broker 110 mayalso be referred to as an Operations Management Platform (OMP). In thisfigure, MEC refers to Mobile Edge Computing.

Each of the devices described above may be implemented as a hardwaremodule. In some embodiments, this hardware module comprises a processingunit, in communication with a computer readable non-transistor storagemedium. This storage medium may comprise instructions that enable thecomponent to perform the functions described herein. In certainembodiments, two or more of the components described above may beincorporated into a single hardware module, and are implemented assoftware components.

The methods described herein facilitate creating, managing, monitoring,learning and auto-tuning Secure Network Slices that carry micro andmacro application services in the evolving 5G Network Paradigm. Both theend devices and network partners will have certificates (specificallyeDCs, which are X.509 Digital Certificates with the enhancementsidentified herein) on their devices, assigned by the enhancedCertificate Authority(eCA). The metadata of the enhanced digitalcertificate (eDC) dictates the provenance and other metadata attributesthat are supported. The metadata includes, but not limited to:

-   -   (a) Hardware or Software Make    -   (b) Hardware or Software Model/Release    -   (c) Region where device is manufactured    -   (d) Location where device is manufactured    -   (e) Other Provenance Requirements    -   (f) Priority/Service Class    -   (g) QOS parameters (latency, bandwidth, reliability)    -   (h) Service duration    -   (i) Service criticality, redundancy and availability    -   (j) Network device type (Bridging, Switching, Routing, VXLAN,        MPLS)    -   (k) Types of routing (Source Routing, Hop-by-hop routing)    -   (l) Hierarchy levels of network slices (sub-ordinate network        slices with subset of QOS and Policies) for micro services. The        subordinate slices may use different QCI levels within RAN with        dedicated RABs for certain QOS levels.    -   (m) Device operation and Certificate type (User/Client, network        edge, operator edge (connects two operator networks, and the        operator names)

One such eDC is shown in FIG. 3. Of course, additional metadata may alsobe incorporated into the eDC, and the disclosure is not limited to onlythe listed metadata.

The metadata is intended to include both the Branded Device Modelinformation, and the Original Manufacturer information that manufacturedthe brand; for example, Company A in China manufacturing products forNOKIA in USA. While the basic X.509 digital certificates with standardattributes are installed during manufacturing, different portions of themetadata extensions may be onboarded by the original manufacturer priorto shipment or onboarded post manufacturing at a reseller location, atan operator procurement site, or in device operational location incoordination with the trust assured entities. The method of onboardingthe entirety or subset of the above metadata onto Digital Certificatesat different stages before the device is put into operational use tocarry SNS traffic is outside the scope of the current disclosure.However, the metadata extensions to X.509 certificates for enhancedsecurity and trust and using these certificates for Secure NetworkSlices (logical networks with enhanced security and trust) are disclosedherein.

FIG. 6 shows the PKI infrastructure components that may be deployed tosupport the enhanced digital certificates. As noted above, the enhanceddigital certificates comprise policy object identifiers to allow for endto end policies across a multi-domain network. The policies arespecified for a plurality of security levels in the enhanced digitalcertificates as object identifiers. In this embodiment, there areclients, 120, 121 and a server 122. Between these devices is a 4G/5GRAN/Core 123, an enterprise SDN 124, an SD-WAN 125, and a public and/orprivate cloud 126. The internet 127 may also be accessible via thisnetwork. For simplicity, it is assumed that there is one network element(NE1) disposed in the 4G/5G RAN/Core 123, one network device (NE2)disposed in the enterprise SDN 124, one network device (NE3) disposed inthe SD-WAN 125, and two network elements (NE4 and NE5) disposed in thepublic and/or private cloud 126. Certain network domains may containtheir own Security System Components (CA, TA, RA and others). In FIG. 6,the 4G/5G RAN/Core 123, SD-WAN 125 and public and/or private cloud areshown as having domain specific PKI components, labels SC1, SC2 and SC3,respectively. Of course, more or fewer network domains may contain thesedomain specific components. Further, one or more multi-domain SecuritySystem components, labeled MD-SC1, MD-SC2 and MD-SC3 may also bedisposed in the network, and these components may be accessible tomultiple domains. The multi-domain security system components, labeledMD-SC1, MD-SC2, MD-SC3 support enhanced policies across the multi-domainnetwork. The policies are specified as object identifiers in eDCs. FIG.6 shows MD-SC1 and MD-SC2 located in secure locations, and MD-SC3 as avirtual system deployed in Cloud. The multi-domain security componentsinclude Enhanced CA (eCA), enhanced OCSP Server, enhanced OCSP proxy andothers that are involved in the life-cycle of eDCs. FIG. 10 shows anexample showing OCSP Server/Proxy in the AMAZON AWS.

Specifically, FIG. 10 is an example deployment configuration with theend to end Multi-domain security system components that support themethods disclosed herein in an Amazon AWS Cloud Environment. The OCSPPlatform 1000 and the Certificate Platform 1001 shown are softwaremodules (VNFs) that use the methods disclosed herein and useinfrastructure HW and SW services. These platforms work in coordinationwith the Multi-Domain Security System components in physically securelocations. The OCSP Platform 1000 and the Certificate platform 1001maintain shadow copies of certificate related information such asrevocation, validity periods and other parameters, to reduce end to endround trips for certificate retrieval and validations to achieve “NearZero Latency (NZL)”. Using Analytics/Learning Algorithms andopportunistic shadow copying using underlying OS to locate certificatedata close to the endpoints that use the specific certificates, theseplatforms reduce round trips and latency. These shadow copies may bemanaged by the OCSP proxy methods within the OCSP Platform 1000 and theCertificate platform 1001 shown in the figure.

In certain embodiments, security system components such as theCertificate Authority (CA), the OCSP Server/Proxy and others, may beenhanced to support the disclosed methods. The term “eCA” refers toenhanced Certificate Authority that validates identify of requester,either end user, a business entity or an intermediary, such as a IOTGateway, and issues eDCs. Furthermore, these enhancements may includeassuring, validating and trust assurance that the subject identified inthe enhanced certificate meets metadata attributes such as manufacturelocation, region and others. This validation may be by the OEM orassembly location or at a service provider procurement location byinteraction with the trust assured systems at the manufacturing orassembly locations. For example, all of the physical or virtual devicesor software modules that are involved in setting up or using a SNS mayrequire trust assurance via eCAs.

Although not shown, each network domain and cloud operator uses its ownNetwork Controller and Element Management System (EMS). The end to endmulti-domain security infrastructure defined herein uses its ownController/Orchestrator (see Central Broker 110 in FIG. 1B) to interactwith each domain and cloud controller, ONAP, or Element managementsystems in order to set up the “Secure Network Slices”. In this way, itis able to determine the compliant transit network devices.

The policies required for a network slice are dependent on the specificapplication or service, and endpoints that use the network path. Forexample, an IOT Gateway that uses 5G Mobile network as an uplink thatservices several healthcare IOT devices in an assisted living facilitymay require periodic reporting to a healthcare provider and may needhigh bandwidth connectivity to local health care provider (hair pinningat the edge) during emergencies. A network slice that supports streamingservice from a stadium via 5G network requires high bandwidth transportto plurality of application servers during an event at the stadium.

Additionally, for supporting network slices at multiple security levels(for example, Low, Medium, High), policy requirements for each level maybe different. For example, a network slice with SL2 security level mayrequire nailed down transit paths through each device, maximum BW,exclusive interface use, no mirroring/replication at interfaces, andsource routing or static routing instead of hop-by-hop routing, minimaltime of presence in the transit network to minimize securityvulnerabilities and other constraints. Additional polices for SL3 flowsmay include limiting the active duration of slice or data flow throughthe slice without revalidation of security policies, allowing onlycertain P2P protocols, not using alternative path or pre-validated pathonly when an interface goes down and other constraints to minimizesecurity compromises. The security policies listed above are examplepolicies; specific use cases may require other policies.

Network elements that support the security policies described herein arevalidated by testing in a test environment with specific vendor's devicemodels and software versions. The enhanced digital certificates areissued by offline or online methods. The methods of obtaining acertificate are well known and are not described in this disclosure.Enhanced certificates are incorporated into these devices via offline oronline methods prior to dataflow through the network slices.

Additionally, the network element is required to support configuringparticular interfaces via management and configuration methods byidentifying the next hop devices that they connect to. The interface ismarked with a security level (such as SL1, SL2, SL3 and others) based onpolicies to which they are compliant. In each transit network domain,all possible forwarding paths from one network-domain endpoint to thenext network domain endpoint are determined, and each transit device andits ports may be marked with Security level. This process is continuedacross all the domains in the multi-domain network from each of themulti-domain network end points. For example, this process is performedat enterprise locations that connects to the multi-domain network atmultiple geographical locations.

While setting a transit path, which may be done either via routingprotocols or by configuring transit network forwarding paths using adomain controller, only interfaces/ports that are marked with thecorresponding security level are used. If the network slice operates atL2 protocols, only compliant interfaces/ports are enabled to theforwarding state. If the multi-domain network contains virtualnetworks/VNFs, the same process is used for these elements.

The network elements (transit or endpoint) may be a chassis withpluggable line-cards (hardware modules) from different vendors. Whileselecting a NE as compliant with one or more levels of security, thevalidation method needs to ensure that specific line cards are compliantwith the security policies (for example, the line-card is not fromprohibited vendors identified in the enhanced digital certificate) andmay use per linecard certificates.

While the setup of SNS's may be performed using a central broker 110,the network elements in the path may also play a role. For example, aPhysical/Virtual Network element, referred to NE-A, may, beforetransmitting on a physical/virtual/cloud network interface or a networkslice or TCP/UDP session, to a second network element, referred to asNE-B, validate that NE-B has a digital certificate with the neededpolicies to receive data from NE-A and that it supports the policies perthe certificate.

Further, a compliant NE-B that is intended to receive network flows fromNE-A in turn validates the authenticity of NE-A, before processingflows. While processing flows, if NE-B needs to transmit the data(forwarding or after flows processed by local applications) to pluralityof outbound interfaces, it performs the steps described above withrespect to NE-A.

On the other hand, NE-B may be a consumer of the data flows from NE-A.For example, NE-B may be a database server, or protocol/applicationproxy etc. in the enterprise physical or virtual network or a VNF. Inthis embodiment, NE-B is processing a received request, the requesteddata in NE-B may require a different set of security on who/whichsystem/client could request and what the transit NEs should assure. Inother words, data traversal in the reverse direction may requirerevalidation similar to the above steps. In such a case, the respondingnetwork element may reject the request, indicating insufficient securitypolicy, thus causing the originator to move to a higher level of networkslice or orchestrate the reverse flows to use a higher level of networkslice. As an example, a request for data may be made to a databaseserver using a SNS having a first security level. When the databaseserver obtains the requested data, it may discover that this data mayonly be transmitted using an SNS having a higher security level. In thiscase, the database server will reject the request and indicate that therequest must be made using a higher security level.

It is noted that at boundaries between networks, for example, fromenterprise SDN 124 to SD-WAN 125 or cloud provider 126, as shown in FIG.6, the provider may not provide visibility to internal physical orvirtual network elements. In such cases, the security policies mayrequire performing the above steps at these boundaries.

In this way, data integrity, security certificate validation is achievedat each physical or virtual hop and termination point in the path ofnetwork slice. This includes AWS, EPC, and other service providernetworks. In other words, all systems along the flow path may providerepudiated slice integrity to flows.

In certain embodiments, the network elements may label their interfacesbased on the security level. For example, for Security Level 1 (SL1),which is the default, low security level, the network elements mayfollow the usual destination forwarding/L2/L3 rules. However, thenetwork element may determine that, in the paths, no NE will be fromVendors “X”, “Y”, or “Z”. Thus, an enabled NE may mark each interface as“SL1”, depending on to which device it connects. If the interfaceconnects to a device from vendors “X/Y/Z”, it will not be marked as SL1.In all the devices that have the enhanced Digital Certificate, all theports marked “SL1” would be an “eligible port group” for any L2/L3Forwarding.

A medium level, referred to as Security Level 2 (SL2), may includeeverything contained within SL1, but may disable port mirroring andflooding, for example.

A high level, referred to as Security Level 3 (SL3), may includeeverything contained within SL2, but force nailed down routing, suchthat upon interface failures, the traffic will not be rerouted.Additionally, the device may verify the path at the start of a new flow.

Of course, the description above is purely illustrative and the securitylevels may be defined differently.

In the edge device, such as the CPE at an enterprise location thatinterfaces to the CSP that connects to the slice, the device will have acertificate for each slice (Slice Certificate) indicating the slicerequirements. For example, the R&D dept in an enterprise may requirehigher security level (SL3) than Customer Service (SL1). Thus, the eDCfor SL3 may require only Vendor A device and Safari Browser only, andSL1 may allow devices from Vendors A,B,C, and Safari and Chrome. In theenterprise location, the Authentication/Authorization server validatesuser device, user access profile, user department, and determines thewhich security level slice to connect to.

3GPP Standards define network slices in 4G/5G networks with QOSparameters for wide variety of applications and micro-services (such asIOT, M2M, Vehicular Communication, voice, internet, video and others).The present disclosure describes using different security levels andassociated assurances in addition to the QOS parameters. For example,SL1 (Low Security Slice) may use the default bearer in 4G/5G Network,and SL2/SL3 may use dedicated bearers. A wireless mobile operator orMVNO that offers these security assured services may use dedicatedbearers with different QCI and protect that traffic via a dedicated userplane GTP-U tunnel while carrying the traffic through the mobilenetwork. For example, each secure network slice may terminate in adifferent APN in mobile core network.

In certain embodiments, one or more mCAs maybe employed. The term “mCA”refers to a Mobile Native Certificate Authority which incorporates themethods described herein. The mCA incorporates Control Plane and UserPlane protocols and procedures to operate in mobile network to minimizelatencies. This may be a new hardware module that incorporates subset ofthe methods described herein or a software module that is installed oncommodity hardware platforms with layered software (Host OS,Virtualization Software, Containers etc.) or in Public/Private Clouds.Specifically, the UP protocols use GTP-U tunnels where user IP packetsare encapsulated in transport layer IP packets. The transit network mayselectively forward flows targeted to mCA or mCA may recognize andprocess flows targeted to itself, terminate GTP-U tunnels, extract userIP packets, and respond to user requests. Other security systemcomponents such as OCSP Server or Proxy OCSP server, may selectivelyextract user IP packets from encapsulated GTP-U tunnels and process thetargeted flows.

The process of connecting a mobile device to a secure slice is describedbelow.

First, the mobile device connects to operator network in the controlplane. The operator allows this connection based on recognizing user anduser device, based on authorization and authentication, which may beperformed using IMSI, IMEI. In other words, the eDC may not be used atthis time.

This is essential for establishing any UP/GTP-U tunnels. The user thenrequests to setup UP/GTP-U tunnels for a specific application, whereinthe QOS is specified while connecting. The user also specifies APN. TheAPN could be different for different security levels.

After the GTP-U tunnel is established, all the user data from thatapplication (both eCA data and other slice traffic) travels through thistunnel.

In prior art environments, the security platform components such as theCA, OCSP Server, OCSP Proxy and others that participate inauthentication of user plane sessions, are not in the mobile network,and are beyond the PGW. In other words, these components do not processuserplane traffic in tunnels. Thus, the PGW terminates GTP-U Tunnels,and forwards user IP payload to Internet or other targeted systems.Thus, in prior art systems, the CA processes user IP packets and notGTP-U tunneled packets. This will have path latency to PGW, and in thatthe CA is not in mobile network, either the radio network or the corenetwork.

Thus, in one embodiment, security platform components such as eCA, OCSPServer, OCSP Proxy and others, are disposed in the mobile network.Placing an eCA or eCA network in mobile network may require terminatingGTP-U tunnels for that user and processing all user IP flows—(thishappens in Mobile Edge Computing (MEC), where the virtual PGW terminatesentire GTP-U tunnels), and would have to do various proxy and forwardingfunctions for sessions it cannot process.

Alternatively, from the GTP-U tunnels of a user, only flows targeted atsecurity platforms such as the eCA are forwarded by the underlyingnetwork, and the enhanced security platform terminates the GTP-U tunnelsand processes flows targeted to itself. Network devices such as DPI(Deep Packet Inspection) Boxes, or network element with deep packet lookup based selective filtering, forwarding or replicating capabilities, ora SIPTO (Selective IP Traffic Offloading) device in operator networksperform these forwarding operations. Similar to eCA, other enhancedsecurity system components, such as OCSP Server or Proxy that maintainscertificate expirations, revocation lists and other items, couldterminate GTP_U tunnels and process received flows.

The policies of a network slice may allow setting up hierarchicalslices, in the sense that for a new slice QOS, the policies, such assecurity policies, may be within the scope of previously set up slice.For example, a network slice setup to an Assisted living facility maysupport plurality of services, such as Video Streaming, Internet,Messaging for carrying IOT traffic from medical IOT devices. An IOTgateway may initiate a periodic and/or a need-based network slice withassociated QOS and policy parameters. The underlying plurality ofnetworks (WIFI, LTE, LTE-M, LTE-NB, Wireline, cloud and others) may usethe network paths most appropriate for such service.

Slice selection may be performed in a number of ways. When an enterpriselocation that uses an edge network element, such as a CPE router,Firewall or other device, to access multiple secure network slices, eachslice with a particular security level forms a virtual network. If theenterprise location has multiple departments, each department mayrequire a different security level, map to a different subnet, and use aspecific slice through the virtual network. Traffic from a client devicein that organization goes through the network slice with the associatedsecurity level. Alternatively, specific network slice selection may bedriven by a client device or client application selecting a slice withspecific security level while opening an application session. The sliceselection may use additional validation techniques such as Multi-factorauthentication, EAP, CAPTCHA and others.

Alternatively, instead of setting up network slices with multiplesecurity levels before use, a client device using a default or lowsecurity level slice, may trigger to set up a new slice with highersecurity level, in a manner similar to creating dedicated bearers inmobile networks (UMTS, LTE), or in setting up VPNs. It is envisionedthat transit network domains (Carrier, SDN-WAN, Cloud and others) needto provide different guarantees and framework for different securitylevels and charge higher tariffs based on security level, use-time,latency and bandwidth. Applications may use higher security level slicesonly when needed to minimize costs.

In other words, it is envisioned, that policy assurance for differentsecurity levels may require different resource levels, prioritization,validation, and network service providers may charge differently fordifferent security levels. Thus, clients of a specific network slice mayminimize the use of high security level only for critical datatransfers; for example, for backup or replication of sensitive data fromserver in one location to another for protection, or to minimize accesslatencies.

While setting up sessions through secure network slices, the networkendpoints of the network slices may validate that the clients andservers that use the network slice are at the required security levelfor that slice. The method of validation may be via the client andserver IP Addresses, authentication mechanisms (such as EAP),determining the departments they belong to, user and clientauthentication via multi-factor authentication, captcha tests and othertechniques.

In certain embodiments, path validation while setting up a securenetwork slice or during use may be via control/management planes byinteracting with domain's PSF/VSF systems, ONAP, service providersControl and Management systems or in-band via extensions to protocolssuch as TLS. This validation may be done on a hop by hop basis so thatevery segment in the transit multi-domain network (MDN) is compliantwith certificate policies.

In summary, network elements, such as switches, routers, bridges, SDNDevices, Virtual Network Elements and others, in the transitmulti-domain network may contain their own vendor unique digitalcertificates. Additionally, if the device is a chassis with pluggablelinecards, the linecards may contain their own vendor uniquecertificates.

This disclosure describes the incorporation of vendor independentenhanced digital certificates with enhancements for Secure NetworkSlices to create compliant devices and linecards in coordination withthe vendor at factory or at operator procurement or enterprisedeployment locations.

FIG. 7 shows network dataflows of network slices through multiplesystems. The policies required per particular security level in anenhanced Digital Certificate may be different on North/South interfaces700 that connect the network edge to the MDN Network and East/Westinterfaces 710 that connect the network elements in the transit network.FIG. 7 also shows Application Servers (VNFs) 720 such asWeb/Mail/Database servers, in the cloud that terminate flows connectingto North/South interfaces 700. As stated above, the metadata attributesand the associated security level required may be different fordifferent flow directions, such as East/West directions 710 on packetforwarding NEs and North/South interfaces 700 that process at higherlevels up to application level as in Servers and Proxies.

The transit network between multiple locations of an enterprise thatrequires high security may include multiple service providers, virtualnetworks, cloud networks, and each such network contains multipledomains, physical and virtual network elements, as shown in FIGS. 1B, 5and 6. Additionally, the physical/virtual elements in the multi-domainnetwork may include application services (Hypervisors, VMs, Containers,Servers, VNFs). Thus, the present disclosure describes DigitalCertificate enhancements with object identifiers for plurality ofsecurity levels with metadata for all of these transit Network elements(physical, Virtual, Cloud, VMs, Hypervisors, Containers, VNFs, EndpointsClient Devices, Client Applications, Servers, proxies, VNFs). Further,the present disclosure ensures that each network element holds thecertificates with required security level for secure traffic in eachdirection (North/South, East/West).

The certificate enhancements may be dependent on the type of physical orvirtual network elements, interface type (North/South, East/West),security level and other parameters. This approach allows integration atthe OEM supply chain, the acquisition process where one carrier acquiresanother carrier's assets, or even an Enterprise that wishes to harness avariety of network platforms to obtain end-to-end service.

While selecting a transit network device for inclusion in the path, eachof the network devices, and the linecards that may carry the securenetwork slice traffic need to be assured that they hold the enhancedsecurity certificates. The robotic process assurance using enhancedcertificates may be achieved, for example, by identifying all thetransit IP addresses between the slice termination points (edge networkelements) by utilities such as “trace route-all”, determining that eachdevice corresponding to that IP address (chassis) holds the enhancedcertificate, checking each transit forwarding element (L2 forwardingswitch/Bridge, MPLS switch etc.) between two Layer 3 devices and thenchecking which linecards hold the enhanced certificate for inclusion inthe secure slice path. Additionally, for inclusion of a networkinterface for slice traffic, the next hop connected device and theassociated linecard need to be checked that they hold the enhancedcertificate. The process of checking that each device and the pluggedlinecards hold the enhanced certificates, and are compliant withsecurity policies may be achieved using the correspondingPhysical/Virtual Control, Management, Security platforms (PSF/VSF) orEMS, ONAP, and/or management protocols. This process needs to berepeated across all transit network devices hop by hop, and thecorresponding interfaces marked as eligible. Additionally, since thelinecards may be hot-swappable, and cable connections between thedevices may be changed by service personnel, a subset of the processneeds to be performed when an interface comes up, or after hot-swap of alinecard or device reboot, or when a virtual service is moved from oneunderlying physical or virtual component (HW, VM, Container) orexpanded/distributed for redundancy, backup, or for increased load.

As noted above, FIG. 1B discloses a Central Broker 110. The centralbroker 110 is used in combination with security system components suchas the CA, the TA, the RA, and others in the multi-domain network. Thecentral broker 110 understands the enhanced policies; managescertificate validation; manages operations using Shadowing/Mirroringetc., and manages features using physical or virtual/cloud elementsdistributed in the network to minimize round trips and end to endlatency.

This central broker 110 may be implanted in a number of ways.

For example, the central broker 110 may be a SNS-OMP 501 (Secure NetworkSlice Operations and Management Platform), as shown in FIG. 5. In thisembodiment, the SNS-OMP 501 sets up network slices either from userinitiated requests or enterprise or operator initiated requests. TheSNS-OMP 501 discovers the available networks of plurality of serviceproviders, determines transit forwarding paths from each of the serviceprovider networks, determines attributes of each network element(Manufacturer Name, Model Number, Software Version etc.), and determinescompliant network elements (Network Elements that support the policyrequirements). This may include interacting with a controller 502 inOperator Network A, a controller 503 in Operator Network B, andcontrollers 504 in the cloud networks through which the secure networkslice traffic traverses.

The central broker 110 (which may be SNS-OMP 501) identifies additionalconstraints based on the metadata in the enhanced digital certificatewhile using source routing or hop by hop routing or L2 forwarding duringsetup of network slices so that malicious network elements or virtualnetwork functions do not compromise security. The transit network pathsmay use Layer 2 or combination of Layer 2 and Layer 3 segments. In layer2 devices, only interfaces that connect to compliant next hop devicesare selected as eligible ports.

The following illustrates several embodiments by which a secure networkslice may be established. Reference is made to FIG. 5.

In one embodiment, an enterprise that requires SNS paths to its multiplegeographical locations uses a central broker 110, such as SNS-OMP 501that interacts with service provider Operation Management Platforms(OMPs) such as OSS/EMS, ONAP to determine transit network elements, andqueries each device to identify devices that hold eDCs. The OMP is alogical entity that includes one or more servers, such as authorizationand authentication servers, such as RADIUS, DIAMETER, policy server andothers that validate user access privileges, authorize user devices andperform other functions. Each Service Provider OMP via APIs providesmethods to determine their network interconnect topology, routing andforwarding paths, and to mark, and configure routing paths (for example,static routes). SNS-OMP 501 uses client applications (such as ONAPClients) to interact with these operator OMPs.

FIG. 5 shows an example network that illustrates SNS setup. The CPEs 505shown are customer premise devices that connect the enterprise networkto one or more communication service providers, for example, COMCAST,Verizon, ATT etc. The dotted lines in the figure show logical controland management paths between the SNS-OMP 501 and service provider OMPs.

The following steps illustrate setting up SNS logical network accordingto a first embodiment. The steps are illustrated in FIG. 8.

In certain embodiments, the enterprise requires 3 different securitylevels (SL1, SL2, SL3 where SL3 is highest security level), for 3different departments, such as Sales, Customer Service, and Engineering.In this example, the security policy requires only Nokia ND1 models withsoftware version V10 for SL3, and ND1 devices from Nokia, or CD1 fromCISCO, or ED1 from Ericsson are allowed for SL3 traffic.

The Enterprise may purchase network services from Access Network ServiceProviders, such as Operator-A Network, and Operator-B Network, SDN andpublic/private clouds, for network and hosting services (Applicationservers, Database Servers, caching and proxy servers). As part of theservice, the enterprise negotiates access to each of the serviceprovider's OMP (OSS/EMS, ONAP etc.). The Access Network Providerprovides one or more IP addresses for forwarding traffic to/fromenterprise locations. These addresses terminate in the CPEs 505 inenterprise locations. Thus, SNS paths terminate in the CPEs 505 similarto CPE deployments in prior-art deployments.

For setting up the SNS at security level SL1, SNS-OMP 501 uses“traceroute all”, to determine all possible paths from local CPE toremote CPE. The “traceroute all” response includes transit IP addresseswithin multiple SP networks in the transit network. The forwarding (L2or L3, MPLs etc.) paths through internal network elements within eachservice provider's internal network may be unknown from the responses.

From the “traceroute all” responses, the SNS-OMP 501 identifies transitL3 routing element addresses that correspond to L3 routing NEs inmultiple SPs. The SNS-OMP 501 interacts with each SP OMP 502, 503, 504to determine the network elements that terminate the addresses, andwhether the devices hold eDCs, and verify each device and/or virtualentity is trust compliant. The SNS-OMP 501 marks only those devices thathave eDCs as allowed devices to carry SL3 traffic.

Corresponding to each transit and the next hop IP addresses, the SNS-OMP501 queries the corresponding SP OMP to determine transit nodes (L2, L3etc.) between the two addresses. The SP OMP or the SNS-OMP 501 incooperation with SP OMP could use “traceroute all”, LLDP, SNMP or otherprotocols to determine transit nodes.

In the previous steps, the SNS-OMP 501 identifies all the transitdevices that are allowed to carry SNS SL3 traffic.

The SNS-OMP 501 then interacts with each SP OMP to setup static L3route, L2 or other forwarding paths in its network up to the entry pointin neighboring SP Network.

As an alternative to setting up a static route in each service providernetwork, the SNS-OMP 501 in cooperation with SP OMP configures next hoplinks through allowed devices of each transit SP network.

The following steps illustrate setting up SNS logical network accordingto a second embodiment. In this example, the transit SPs set up a SNS intheir own networks, and make compliant transit route maps visible to theSNS-OMP 501, thus not providing support to set up transit paths orvisibility to all of the network elements in their network. Thisfacilitates the client enterprise to access and validate physical andnetwork paths already setup by the SP, thus isolating SNS's of multipleenterprises.

In this embodiment, the Enterprise uses SNS-OMP 501 as shown in FIG. 5at one or more operations centers which interacts with SP OMPs usingAPIs to ONAP, NETCONF, SNMP and other protocols. Additionally, theSNS-OMP 501 validates user and device authentication/authorization andsecurity level for specific departments when application sessions areinitiated through CPE 505 for using SNS paths.

For setting up a SNS at a specific security level (e.g., SL3), theSNS-OMP 501 determines entry point address (e.g., default gatewaya.b.c.d) in the access network.

The SNS-OMP 501 then determines transit L3 routes from local CPE 505 todestination CPE 505 or Physical/Virtual server in the cloud to determinea plurality of transit forwarding paths through multiple serviceproviders by commands such as “traceroute-all” which returns multipleroutes to destination and transit L3 addresses on each route. However,such response does not include L2 switch/bridges between two L3 nodes.L3 addresses in each routing path facilitate determining transitnetworks.

The SNS-OMP 501 then connects to each OMP corresponding to each SPidentified above to find transit NEs in each SP network. In coordinationwith the SP OMP, only trust assured paths (NEs with eDCs) are marked asallowed for the specific SNS.

In coordination with OMPs of transit SPs, the SNS-OMP 501 orchestratesredundant paths, in each SP network and at the SP boundaries. Forexample, at one location, there may be 2 CSPs providing networkconnectivity to next SP networks or Cloud. Setting up multiple networkpaths for redundancy and availability is well known in the prior art.While setting up such paths, the SNS-OMP 501 ensures such alternatepaths include only allowed network elements (with eDCs).

The enterprise also negotiates with each transit SP for automaticallynotifying the SNS-OMP 501 of any link or network element failures, forexample via SNMP traps.

Thus, in this embodiment, the SNS-OMP 501 effectively stitches togethera complete Secure Network Slice based on partial secure slices that areprovided by the OMP of each service provider.

Upon receiving path failure notification, the SNS-OMP 501 selects thealternative path learned above, revalidates trust compliance andorchestrates using alternate path via the respective SP OMPs.

The following steps illustrate setting up SNS logical network accordingto a third embodiment. In this embodiment, the SNS-OMP 501 interactswith Access Network CSPs to determine physical or virtual network pathsto determine entry and exit paths to/from access network to next segmentproviders from one enterprise location to other. For example, ifOperator-A network offers connectivity to the SDN, the operator-A OMP502 provides alternative paths (corresponding network element identitiesor addresses), so that the SNS-OMP 501 could determine or orchestratetransit path to alternate SDN entry points.

Starting with the first entry point device that connects to CPE 505 atthe first enterprise location, the SNS-OMP 501 determines which deviceshold eDC by interacting with the Operator-A network OMP 502. The SNS-OMP501 marks each physical or logical interface between the CPE 505 andallowed Operator-A network entry-point device as “allowed” for thatsecurity level, for example SL1.

The SNS-OMP 501 then traverses physical or virtual interface paths fromthe allowed entry-point device through each connected physical orlogical interface to determine if the connected device holds an eDC. Ifthe connected device holds the eDC with the specific metadataattributes, it marks the interface as “SL1-Allowed”. This step isperformed through all the devices between the entry point to the exitpoints from CSP1 to the entry points of the next segment provider, forexample SDN1.

The SNS-OMP 501 performs the above steps to mark all interfaces as“SL1-Allowed” if the device itself and the device the interface connectsto holds eDC. This process marks eligible interfaces as “SL1-Allowed” ineach transit service provider network.

The SNS-OMP 501, in coordination with SP OMP in each transit network,may create a unique VLAN Tag for the SL1-Allowed interfaces, so that L2VLAN methods use only those ports.

The SNS-OMP 501, in coordination with SP OMP in each transit network,may create a virtual interface and logical IP subnet on each of theSL1-Allowed interfaces, so that forwarding paths to the destinationenterprise endpoints may be specified through these logical interfaces.

The following steps illustrate setting up SNS logical network accordingto a fourth embodiment. An Access Network provider, such as VERIZON orATT, may offer managed SNS service at multiple security and servicelevels (SL1, SL2, SL3), performing the SNS setup end to end, providingonly trust validation methods via APIs to its OMP. This service issimilar to VPN services offered by CSPs with the difference that VPNservices use end to end encryption at the two end points without trustassurance of each network element that carries the VPN traffic. SNSsetup may be achieved by CSP1 using SNS-OMP as in the previous examples.The CPE (or virtual CPE) that connects to the network at enterpriselocation terminates the SNS.

Of course, these are illustrative examples and the disclosure is notlimited to only these embodiments.

The following are some example profiles that are enabled by the presentdisclosure.

In the Open Network Automation Platform (ONAP) environment, the networkslice selection policies (NSSP) would be dictated primarily by thisprofile.

In one profile, communication over the partner's network is only routedthrough equipment sourced to friendly country suppliers. In other words,the security level may require equipment only from a select group ofcountries. For example, this would NOT allow traffic to be routed overequipment or software that is unfriendly to the country's strategicinterests. Only validated and trusted equipment are allowed in thetraffic paths.

In another profile, communication over the partner's network is onlyrouted through equipment that is made in a particular country, such asthe United States.

As an example, FIG. 4 shows allowed and dis-allowed transit paths for aSecure Network Slice with a specific profile, according to oneembodiment. Certain transit network elements 401 do not hold the eDCs.For example, these network elements may not be from a qualified vendoror may not be manufactured in an allowed region. They are marked with“X” indicating those network elements 401 are not allowed to carry thespecific SNS traffic. The authorized network elements 402 hold an eDCthat supports the policies for that particular slice and are allowed tocarry SNS traffic.

While network paths are normally determined by routing, bridgingalgorithms, or by a network Controller/Management Platform based on QOS,the present disclosure uses additional constraints for assuring securityand trust. The users or enterprises that use the network slice coulddictate which vendors, models, software versions, transit virtualnetworks, cloud frameworks, geographical locations etc., that theytrust.

The following example shows the steps undertaken to set up a networkpath.

First, a client device, such as a mobile/IoT device, needs a connectionto “somewhere” past a wireless AP or 5G RAN. The user or operatorselects a profile (which would dictate which certificate will beacceptable) or the device or the application of the device is“hard-coded” for a specific profile.

Next, the client device authenticates to first hop (WiFi or 5G RAN) thatit holds the required eDC. While allowing the service, for example, thecontrol plane (CP) connection to the operator network, and/or allowinguser plane (UP) tunnel to a secure APN/PGW, the operator networkauthenticates the user device and authorizes user credentials. Thisauthentication may check, for example, a specific manufacturer of amobile phone, the version of the phone, the software revision or otherparameters. This may be done using an EAP-TLS RADIUS server for WiFi orDIAMETER for a 5G RAN.

The mutual Transport Layer Security (TLS) allows the client device toalso ensure the host has the proper profile metadata, meaning it cansupport the requested SLA (Service Level Agreement).

The device may also authenticate, using the same certificates, toweb-based clients using OAuth2 mechanisms, which are industry standardprotocols for authentication.

The backend infrastructure selects the proper traffic route based uponprofile specified in enhanced digital certificate. For example, when auser device or an application in an enterprise location initiatesnetwork connectivity to a remote server through CPE, the CPE validatesthe authenticity of the user device and the security level from thedevice certificate and uses the SNS at that security level for thecorresponding route.

Specifically, the backend infrastructure does one or more of thefollowing actions:

a. Configures assigned subnet

b. Potentially sets up VPN

c. Sets up a virtual network slice for the profile

d. Selects an existing virtual network slice for the profile

The central broker 110, such as the Secure Network Slice Operations andManagement Platform (SNS-OMP) 501 (see FIG. 5), that is coordinatingslice setup using ONAP, obtains the transit route, and determines ifeach of the transit NEs are compliant with the policies per the enhanceddigital certificate.

Extensions to ONAP, which are beyond the scope of the presentdisclosure, facilitate orchestrating and setting up transit NEs ofmultiple vendors for the policies in the eDC.

Enhancements in the transit network element include the following:

-   -   (a) forming a secure port group in the switch/L2-Bridge/Router        with multiple interfaces based on where the remote switches that        the interfaces connect to. For example, the secure port group        may only allow those interfaces where the next hop NEs have        specific characteristics, such as specific vendors/models/sw        version, or other characteristics and    -   (b) allowing forwarding and route change only to ports within        the allowed secure port group

The physical or virtual ports identified above may be marked with aunique flag, or VLAN (IEEE 802.1Q) indicating the specific port iseligible to carry SNS traffic. For example, an operator may reserve aspecific VLAN for a specific “SNS”, pre-marking eligible ports thatinterconnect devices that hold Enhanced Digital Certificates.

The transit network over which the SNS logical network traffic traversesmay include physical networks or virtual networks that include GREtunnels, VPNs, or VXLANs of network service providers (CSPs) or CloudService Providers. The underlying virtual network layers such as ClientOS, Hypervisor, VMs etc., may be manufactured by a plurality of vendors.Since security compromises in any of the underlying layers couldseverely impact the SNS traffic, it is essential that each underlyinglayer that carries the SNS traffic must be trust assured while settingup SNS for high security levels.

As seen in FIG. 5, the SNS-OMP 501 determines the network paths for aspecific SNS Security level. The SNS-OMP 501 also needs to validate thateach underlying layer in each transit NE holds the required eDC via thecorresponding network control and management platform.

FIG. 9 shows the result of these steps. The enterprise locations 901require secure network slices between the two locations, and also cloudhosted application servers 904. The SNS-OMP 501 (see FIG. 5) interactswith operations and management SNS-OMP 501 extracts geographicallocations of operator network elements and cloud data centers, and thecorresponding device identity attributes, identified as metadataextensions. These attributes could be embedded as eDCs in some devicesor determined by the SNS-OMP 501 while setting up SNS based on requiredsecurity policies.

The following steps show the process of re-validating trust before highsecurity use.

Before using a secure network slice, an end user or enterpriseapplication may want to validate that each NE in the path are trustcompliant.

This may be accomplished by trace route to the destination system (ifthe destination's IP address is known) and verifying that each IPaddress in transit path holds an eDC through the central broker which inturn communicates with the operators' management platforms. Thedestination system may be a physical or virtual network element (VNF)such as a cloud hosted Webserver or Database Server. The trace route maynot list all the local L3 forwarding addresses, or L2 forwarding devicesin the operator network. For example, it may include entry point addressat each service provider boundary, for example, if the slice traversesthrough 3 operator domains, A, B, C, the trace route may only list entrypoint address into A, B, C domains. Within domain A, operator may routetraffic to B through multiple devices. The central broker, inco-operation with operator A's management platform, determines transitNEs to B and validates that each NE is allowed to carry the SNS traffic.

If the end user or enterprise is attempting to connect to a destinationsystem, for example, a webserver using its domain name, the domain nameis resolved to one or more IP addresses using well known DNS methods.After this, paths to each of the resolved IP addresses are validatedusing the technique described above. Finally, a path that meets thesecurity policy requirements is selected.

In current network routing (Layer 3), and bridging (Layer 2)technologies, determining a network that packets traverse are determinedby L2/L3 Network Addresses or other addresses based on the networktechnologies. Physical locations (Geographical Addresses/Regions) ofeach transit NE, or multiple layers of network technologies that apacket traverses are unknown and invisible to the end users orenterprises that use the service.

The network paths may include multiple CSPs, Cloud Service Providers andare not visible to the users using the service. A particular VNF thathosts a server, or proxy in the cloud could maliciously replicate orroute traffic to secure compromised devices, or locations. If thetransit paths and the associated physical/virtual elements (VNFs,Servers, Proxies etc., their locations, or device origins etc.) areknown to the clients, they could explicitly avoid such paths to minimizerisks. Additionally, the risk associated with a network path or networkelement may be dynamic, in the sense that after HW/SW upgrade to adevice, or security holes of a device may come to light after long usageand may be known via public/private media (internet, security agencies,3^(rd) parties and others). The ability by which a client enterprisechooses or avoids transit paths and devices before using SNSsignificantly increases the security and trust of the SNS logicalnetwork.

Currently, there is no way that an end user or an enterprise that dealswith highly classified information can find what physical locations,types of networks (dark-fiber, dedicated Lambda, SONET Transport,G-MPLS, SONET Tributaries (T1/T3/OCN), Cable etc.), that a virtualnetwork or VPN traverses. In high security classified needs, PhysicalPerimeter Security and dedicated physical private networks are used.These solutions are expensive and limited, for example, in using a newsite in a remote area requires building significant infrastructure withhigh cost.

As Virtualization, SDN, and cloud networks are replacing traditionalnetworking, for highly classified enterprises, close to “perimetersecurity” with physical location maps, network types, vendor equipment,and other parameters is of high value. Network Routing is based on IPaddresses, and with dynamic IP, and layering the paths below are unknownto the end enterprises.

A mechanism, analogous to using a map application, with visibility tonetwork location/device, network technology and other features, withpolicy and validation methods for assuring path traversals is essential.The example below explains this use case, and associated requirements intransit network elements.

When user selects a map application, and enters location A in LA, Calif.and B in Boston, Mass., the map application gives alternative routes as:

-   -   1. Local transit roads at A, Highway 90 through Chicago to        Boston, and local locations near B.    -   2. Local transit roads at A, Highway 80 through PA to NJ to        Boston and local routes near B    -   3. It also shows other metrics such as Tolls, Traffic etc.

Considering the above analogy for a network, for highly classified uses,an enterprise needs a Classified Network slice from A to B (such asSL3); location A has 3 entry points, VZ/Fios, Comcast Cable,ATT/Wireless. From each of these entry points, the client wants to knowwhich of the transit network locations are most secure, and decideswhich network types, vendors, equipment etc., carry the traffic, andwhat policies each Network Element (both physical and virtual) assures.The enterprise would also like to use “trace-physical”, similar totrace-route (which shows transit IP address map), that shows transitpaths with NE-Information, and Network Layers, before using network fora “Classified Transfer”. Alternatively, instead of searching physicalroutes, the user enters the requirements in an operator portal, and thepolicy preferences could be contained in an enhanced DigitalCertificate.

To support the configuration described above, a number of steps, whichare outlined below, are required. These steps may include eDCs, metadatain APIs and interactions with a plurality of control and managementplatforms of different service providers.

First, the software module in each NE must support the embodimentsdescribed herein, such as returning underlying layer maps via aprotocol, or via management paths.

Second, there needs to be a software module that forces policies on theunderlying layers, such as no mirroring, no L2 Unknown DA flooding, andforming controlled port groups. This software module defines whichphysical or virtual Links are allowed to be static/dynamic/fail-overpaths for carrying the network slice.

Next, an extension to Routing or Control Protocols to include additionalconstraints is necessary. Currently, L3 routing protocols are based onL3 addresses and lack any notion of other transit methods. For example,in Google Maps, one could select “METRO”, and it shows “walk”, “bus”,“Metro-A”, “Metro-B”, “bus”, “walk”. Similarly, a comprehensive physicalrouting map envisioned in the present disclosure could show the pathwaythrough the transit network, in addition to L3-destination basedforwarding, and the corresponding operators. For example, an end to endnetwork path may include connection to an AWS cloud server from anenterprise location in a remote town. The network path may have thefollowing: internet via COMCAST, Verizon-FIOS in the access network, andVerizon may offer EVPL, or IP-VPN, or MPLS tunnel, Internet Service orother services at different SLAs through their network to connect toAWS. Similarly, a different CSP may offer connection from the remotesite to AWS. The physical routing map envisioned includes thesealternatives.

Next, there needs to be a method of identifying and discovering whichtransit NEs support the previous extensions. For example, the softwaremodules described above could support a query mechanism or this functioncould be performed via management platforms.

Additionally, a Trace-Physical or Trace-All (similar to traceroute asexplained earlier) features needs to be possible.

Further, methods for validation and assurance are needed. Specifically,there may be a software module in each NE that gives copy of each packetand all the interfaces (Physical, Virtual) that a particularnetwork-slice packet went through when required.

For example, the dataflows through network slices may be mirrored orcopied at network edges that connect to the slices or in the transitnetwork below the slice using virtual probes in coordination withnetwork control and management platforms, EMS or VSF and analyzed usingoffline or real-time analytic methods. The methods could characterizethe behavior of a network such as delay, throughput, work load, numberof sessions at the transport level or with application level analysis toorchestrate moving virtual resources, detect problems in the underlyingshared virtual network, (for example, slow response time to a fileserver), DDOS attacks, choosing alternative transit network paths, orscheduling slice usage to different times to minimize securitycompromises.

In addition to specifying the path preferences for the Secure/Classifiedslice, the enterprise may also want to be alerted of path changes thatmight occur in operation.

The X.509 extensions outlined in the present disclosure facilitatetrusted identity of the transit device in multiple dimensions thatinclude hardware and virtualization layers, manufacturing, supply chaininformation and others. End users or an enterprise could extract fullinformation on transit NEs while orchestrating SNS paths. It isenvisioned that the APIs to users or enterprises will have a routeselection option in the security orchestration interface for Ultra HighAssurance (UHAs)+Ultra High Availability (UHAv), which would mean thatthe route would only use equipment and software with the correctlyissued certificate matching the X.509 extension for UHAs+UHAv.

5G Architecture defines network slices with different QoS policies for aplurality of application services, such as IOT, Streaming Video,Healthcare and others, where the service requirements could beinfrequent reliable message, or high BW for streaming during gamingevent. QoS policies are specified in the Control plane (for example, viaS1AP in LTE) while setting up user plane (UP) tunnels (GTP-U in UMTS,LTE). Once a UP tunnel is setup, QoS and the transit paths over whichthe tunnel traverses and UP tunnel terminates (for example, PGW in LTE)is determined. Thus, setting up a network slice with different securityand trust requires adding security policy parameters. These additionalsecurity policy parameters may include, for example, Security Level tocontrol plane protocols and require support by the associated networkelements.

As an alternative, different APNs for different security levels (forexample, SL1, SL2, SL3) may be employed. Since APNs are specified whilecreating UP tunnels, each SL could terminate in a differentphysical/virtual PDN gateway. User applications or enterpriseapplications could initiate different UP tunnels based on the securitypolicy of the application.

5G Network spans several Radio, Mobile-Access and Core Networks withPhysical, Virtual (SDN), and Private/Public Cloud Networks of multipleCSPs (Communications Service providers), Cloud Service Providers, andInternet. PKI infrastructure elements, located in the internet orselective locations in the core network, add significant latency forobtaining and validating certificates. With the exploding variety ofmobile applications and micro services, the need to provide securitywithout significantly increasing latencies is essential. Locating PKIinfrastructure elements such as the CA, the TA, the RA and others, atdifferent portions of the 5G Network Infrastructure, such as in 5G RAN,5G Core, and opportunistically deploying such network elements as VNFsreduces latency significantly. For example, an analysis of thecertification access interactions may be performed to determine where asecurity platform component may be best placed to reduce latency. Forexample, the security platform component may be disposed in the 5G RAN,a network operated by a service provider or an SDN. These elementsassist in faster setup and operational use of a secure network slice.Thus, in certain embodiments, PKI may be offered as a service (PKIaas).Such deployments require that the Mobile Native Security Platform (mSA)element understands the Control Plane and/or User plane Protocols whereit is connected.

Such a deployment may be transparent where it intercepts the mobilenetwork protocol (for example, intercepting GTP-U user plane protocol)and terminate and process Requests to the mSA. Alternatively, operatormay use a specific APN that may use services from mSA. mSA may bespecified as a DNS name that maps to a plurality of mSA's at differentlocations in operator network. Well known DNS methods facilitateassociating multiple physical/virtual servers. Within the mobile networkbelow the PGW, since the UP traffic is embedded in transport layer GTP-Utunnels, transit mobile network would have to selectively terminate DNSpackets that involve mSA, and route to mSA. Alternatively, operator mayuse a designated DNS server for the APN, and tunneled UP traffic to thatserver intercepted. Once specific tunneled traffic is received, the mSAwould have to support extracting the user payload traffic for mSAinteractions. mSAs could be cloud based servers or proxies fordeployment at mobile network edge for latency reduction for securityrelated interactions.

The present disclosure is not to be limited in scope by the specificembodiments described herein. Indeed, other various embodiments of andmodifications to the present disclosure, in addition to those describedherein, will be apparent to those of ordinary skill in the art from theforegoing description and accompanying drawings. Thus, such otherembodiments and modifications are intended to fall within the scope ofthe present disclosure. Further, although the present disclosure hasbeen described herein in the context of a particular implementation in aparticular environment for a particular purpose, those of ordinary skillin the art will recognize that its usefulness is not limited thereto andthat the present disclosure may be beneficially implemented in anynumber of environments for any number of purposes. Accordingly, theclaims set forth below should be construed in view of the full breadthand spirit of the present disclosure as described herein.

What is claimed is:
 1. A method of constructing a transit network pathmap through a multi-provider, multi-domain network that includes eachtransit network element, its provenance and other device attributes,comprising: interacting with a plurality of transit network operationsmanagement platforms to identify attributes of each virtual and/orphysical device; and using these attributes to either: construct aplurality of network paths to a plurality of destination endpoints tofacilitate client orchestrated transit paths; or construct one or morenetwork paths based on required enterprise security and trust policies.2. The method of claim 1, wherein the transit network path map includesgeographical locations of devices, device types, and virtualizationlayers and provenance attributes of each virtual layer or applicationthat transports or processes the packets.
 3. The method of claim 1,wherein the provenance and device attributes are selected from the groupconsisting of device vendor name, manufacturing location, device name,device unique identifiers, hardware and software model numbers, versionnumbers, manufacturing location, country, region, device type andinterface configuration and operational attributes.
 4. The method ofclaim 1, wherein the provenance and device attributes are contained inan Enhanced Digital Certificate.
 5. The method of claim 1, wherein theinteracting with the plurality of transit network operations managementplatforms is performed using SNMP, ONAP, NETCONF and other networkmanagement protocols.
 6. The method of claim 1, wherein the clientorchestrated transit paths are constructed by allowing devices only fromcertain device vendors, models, manufacturing locations, countries,regions, device types and device operational and interface configurationattributes.
 7. The method of claim 1, wherein the client orchestratedtransit paths are constructed to achieve maximum bandwidth, to minimizedata transfer times or to reduce latency.
 8. The method of claim 1,wherein the client orchestrated transit paths are constructed by notallowing devices from certain vendors, models, manufacturing locations,countries, regions, device types, device operational and interfaceconfiguration attributes.
 9. The method of claim 1, wherein the clientorchestrated transit maps and the constructed network maps areconstructed by selecting a type of network path in the transit networkbased on available transit network path segments and device operationaland interface configuration attributes.
 10. The method of claim 1,wherein the required enterprise security and trust policies aredependent on a Security Level of traffic that traverses the networkpath.
 11. The method of claim 1, further comprising interacting with aplurality of network controllers in the transit network to set upforwarding paths to the plurality of destination network endpoints. 12.The method of claim 1, wherein the multi-provider, multi-domain networkcomprises one or more of enterprise networks, mobile networks,communications service provider networks, SDNs, Virtual Networks, andCloud.
 13. A method of constructing network paths that ensure secureflow through a multi-domain network, comprising: allowing only thoseinterface ports in a first network device with multiple interface portsto forward packets based on whether the interface port connects to asecond network device that is a trusted network device; and allowing thesecond network device to receive and process packets based on whetherthe first network device it is receiving packets from is trusted,thereby ensuring the flow is secure in each direction; wherein eachnetwork device is trusted based on the security and trust requirementsof an enterprise for the data being exchanged.
 14. The method of claim13, wherein the each network device is determined to be a trustednetwork device based on an enhanced digital certificate associated withthe network device.
 15. The method of claim 13, wherein security andtrust requirements vary based on flow path and flow direction, such thatsecurity requirements for east/west interfaces may differ fromnorth/south interfaces for each flow direction.
 16. The method of claim13, wherein at least one network device comprises a router, switch orbridge.
 17. The method of claim 13, wherein at least one network devicecomprises a transport protocol termination device selected from thegroup consisting of an application server, application client,application proxy, and application gateway.
 18. A method of minimizinground trips and latency while validating security and trust of enhancedDigital Certificates in multi-domain multi-provider networks comprising:providing a plurality of network devices each with an enhanced digitalcertificate wherein the enhanced digital certificate comprises metadataregarding provenance and other device attributes of the network device;and opportunistically disposing one or more security platform componentsthat support processing enhanced digital certificates within themulti-domain network based on analysis of certificate accessinteractions so as to reduce latency.
 19. The method of claim 18,wherein the security platform components are disposed within a 5G RAN, anetwork operated by a service provider, or a cloud, or an SDN.
 20. Themethod of claim 18, wherein the one or more security platform componentsassist in faster setup and operational use of a secure network slice.